Link prediction with spatial and temporal consistency in dynamic networks

ABSTRACT

A computer-implemented method executed by at least one processor for performing link prediction with spatial and temporal consistency by employing a time-dependent matrix factorization technique is presented. The method includes developing, at a plurality of timestamps, relational data of a sequence of network structures each including a plurality of nodes and learning, by the at least one processor, a feature vector of each node of the plurality of nodes of the sequence of network structures by concurrently optimizing a temporal fitting constraint and a network propagation constraint. The method further includes determining a network structure at each timestamp, determining evolutionary patterns at each timestamp, and predicting links in a future network structure based on an evolution of the sequence of network structures within a user-defined sliding window by reducing time complexities of finding neighbors of each node of the plurality of nodes of the sequence of network structures.

RELATED APPLICATION INFORMATION

This application claims priority to Provisional Application No.62/466,475, filed on Mar. 3, 2017, incorporated herein by reference inits entirety.

BACKGROUND Technical Field

The present invention relates to dynamic networks and, moreparticularly, to link prediction with spatial and temporal consistencyin dynamic networks.

Description of the Related Art

Evolutionary network analysis has become increasingly important inrecent years. One effort relates to temporal link prediction which isemployed to predict a future network structure based on a sequence ofobserved networks. Extensive research efforts have been devoted to thetemporal link prediction task including non-parametric and parametricmodels. The factors in executing a link prediction task in temporalnetworks are spatial and temporal consistencies, which indicate that anode has a higher probability to form a link with a nearby node thanwith a remote node in the near future and that a network usually evolvessmoothly over time. The first factor is akin to that in static networklink prediction and encodes local network propagation constraints from anetwork topology at each timestamp. The second factor globally enforcessmoothness of network evolution over time. Existing approaches, however,seldom unify these two factors to strive for the spatial and temporalconsistency in a dynamic network.

SUMMARY

A computer-implemented method executed by at least one processor forperforming link prediction with spatial and temporal consistency byemploying a time-dependent matrix factorization technique is presented.The method includes developing, at a plurality of timestamps, relationaldata of a sequence of network structures each including a plurality ofnodes and learning, by the at least one processor, a feature vector ofeach node of the plurality of nodes of the sequence of networkstructures by concurrently optimizing a temporal fitting constraint anda network propagation constraint. The computer-implemented methodfurther includes determining a network structure at each timestamp ofthe plurality of timestamps, determining evolutionary patterns at eachof the plurality of timestamps, and predicting links in a future networkstructure based on an evolution of the sequence of network structureswithin a user-defined sliding window by reducing time complexities offinding neighbors of each node of the plurality of nodes of the sequenceof network structures.

A system for performing link prediction with spatial and temporalconsistency by employing a time-dependent matrix factorization techniqueis also presented. The system includes a memory and a processor incommunication with the memory, wherein the processor is configured todevelop, at a plurality of timestamps, relational data of a sequence ofnetwork structures each including a plurality of nodes and learn, by theat least one processor, a feature vector of each node of the pluralityof nodes of the sequence of network structures by concurrentlyoptimizing a temporal fitting constraint and a network propagationconstraint. The system further includes a memory and a processor incommunication with the memory, wherein the processor is configured todetermine a network structure at each timestamp of the plurality oftimestamps, determine evolutionary patterns at each of the plurality oftimestamps, and predict links in a future network structure based on anevolution of the sequence of network structures within a user-definedsliding window by reducing time complexities of finding neighbors ofeach node of the plurality of nodes of the sequence of networkstructures.

A non-transitory computer-readable storage medium comprising acomputer-readable program is presented for performing link predictionwith spatial and temporal consistency by employing a time-dependentmatrix factorization technique, wherein the computer-readable programwhen executed on a computer causes the computer to perform the steps ofdeveloping, at a plurality of timestamps, relational data of a sequenceof network structures each including a plurality of nodes, learning, bythe at least one processor, a feature vector of each node of theplurality of nodes of the sequence of network structures by concurrentlyoptimizing a temporal fitting constraint and a network propagationconstraint, determining a network structure at each timestamp of theplurality of timestamps, determining evolutionary patterns at each ofthe plurality of timestamps, and predicting links in a future networkstructure based on an evolution of the sequence of network structureswithin a user-defined sliding window by reducing time complexities offinding neighbors of each node of the plurality of nodes of the sequenceof network structures.

These and other features and advantages will become apparent from thefollowing detailed description of illustrative embodiments thereof,which is to be read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

The disclosure will provide details in the following description ofpreferred embodiments with reference to the following figures wherein:

FIG. 1 is a block/flow diagram illustrating an input and processing of alink prediction model with spatial and temporal consistency, inaccordance with embodiments of the present invention;

FIG. 2 is a block/flow diagram illustrating an output of the linkprediction model with spatial and temporal consistency of FIG. 1, inaccordance with embodiments of the present invention;

FIG. 3 is a block/flow diagram illustrating link prediction in a dynamicnetwork, in accordance with embodiments of the present invention;

FIG. 4 is a block/flow diagram illustrating a method for predictinglinks of a dynamic network, in accordance with embodiments of thepresent invention;

FIG. 5 is a block/flow diagram illustrating a method for learningfeature vectors of each node of a plurality of nodes of a sequence ofstructured networks by concurrently optimizing a temporal fittingconstraint and a network propagation constraint, in accordance withembodiments of the present invention;

FIG. 6 is an exemplary processing system for performing link predictionwith spatial and temporal consistency by employing a time-dependentmatrix factorization technique, in accordance with embodiments of thepresent invention;

FIG. 7 is a block/flow diagram of a method for performing linkprediction in Internet of Things (IoT) systems/devices/infrastructure,in accordance with embodiments of the present invention;

FIG. 8 is a block/flow diagram of exemplary IoT sensors used to collectdata/information related to performing link prediction, in accordancewith embodiments of the present invention; and

FIG. 9 is a block/flow diagram illustrating a completed dynamic networkby using a link prediction model with spatial and temporal consistency(LIST), in accordance with embodiments of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

In the exemplary embodiments of the present invention, dynamic networksare introduced. One aspect of dynamic networks is link prediction.Models have been developed to predict links that can emerge in theimmediate future from the past evolution of the networks. Linkprediction is based on two factors. The first factor is that a node ismore likely to form a link in the near future with another node withinits close proximity, rather than with a random node. The second factoris that a dynamic network usually evolves smoothly.

In the exemplary embodiments of the present invention, a link predictionmodel is presented with spatial and temporal consistency (LIST), topredict links in a sequence of networks over time. LIST characterizesnetwork dynamics as a function of time, which integrates a spatialtopology of a network at each timestamp and the temporal networkevolution. LIST has at least two advantages compared to conventionaltechniques. The first advantage is that LIST uses a generic model toexpress the network structure as a function of time, which makes theLIST model suitable for a wide variety of temporal network analysisproblems. The second advantage is that by retaining the spatial andtemporal consistency, the LIST model yields better predictionperformance.

In the exemplary embodiments of the present invention, the problem oftemporal link prediction is defined as follows. Given a sequence ofnetworks from timestamps 1 through T, the task is to predict linkweights at timestamp T+α, where α≥1. Note that a special case of thisdefinition is to predict whether a new link emerges or not (e.g., whenthe link weight is restricted to be either 0 or 1).

In the exemplary embodiments of the present invention, a model for linkprediction with spatial and temporal consistency is presented. Thetime-dependent matrix factorization technique is leveraged, which hasshown to be a powerful tool for characterizing dynamic structural data,to decompose the network adjacency matrices into time-dependent matricesthat capture the features of vertices (or nodes) in the dynamicnetworks. At the same time, the network propagation constraint ispresented, which ensures vertices (or nodes) to be within closeproximity to their neighbors in the hidden feature space to be learnedby the time-dependent matrix factorization.

In the exemplary embodiments of the present invention, the featurevector of each vertex is learned by simultaneously or concurrentlyoptimizing a temporal fitting constraint and a network propagationconstraint. The temporal fitting constraint can be expressed as atime-dependent matrix factorization with a network adjacency matrix,while the propagation constraint preserves or maintains similaritiesbetween the connected pairs of vertices (or nodes) in a feature space.The learned feature matrices are parameterized with time and can beemployed to reconstruct the network structure at any given timestamp t.This allows for more general predictions to be made. Additionally, thisfeature matrix can be viewed as a complete profile of the networkdynamics over time, which can also find its utility in other applicationsettings of evolutionary network analysis.

In the exemplary embodiments of the present invention, spatial andtemporal consistency in link prediction is achieved, computationalspeed-up is enabled, and empirical improvements over previous methods isaccomplished.

Regarding spatial and temporal consistency in link prediction, atemporal link prediction model (LIST) in dynamic networks is introduced,which simultaneously or concurrently considers a network structure ateach timestamp and an evolutionary pattern across all timestamps of asequence of structured networks. A network propagation constraint isleveraged, which ensures that connected vertices (or nodes) have similarfeature vectors. This “locality preserving” property captures thespatial structure of networks at each timestamp. The feature vector ofeach vertex (or node) can be learned via time-dependent matrixfactorization across all recent timestamps, which reveals the temporalstructure of networks. Regarding computational speed-up, an efficientalgorithm is developed to learn the LIST model which reduces a timecomplexity from O(n³) to O(mn+n²), where n is the number of vertices (ornodes) in the dynamic network, and m is a number of non-zero entries inthe network adjacency matrix. This makes the LIST model described hereinapplicable to various types of large scale datasets.

FIG. 1 is a block/flow diagram illustrating an input and processing 100Aof a link prediction model with spatial and temporal consistency, inaccordance with embodiments of the present invention whereas FIG. 2 is ablock/flow diagram illustrating an output 100B of the link predictionmodel with spatial and temporal consistency of FIG. 1, in accordancewith embodiments of the present invention.

The feature vector of each vertex or node 20 is learned bysimultaneously or concurrently optimizing the temporal fittingconstraint 10 and the network propagation constraint 12. The temporalfitting constraint 10 can be expressed as a time-dependent matrix 14factored with a network adjacency matrix, while the network propagationconstraint 12 preserves or maintains similarities between the connectedpairs of vertices (or nodes) 20 in the feature space. The learnedfeature matrices are parameterized with time and can be used toreconstruct the network structure at any given timestamp t. This allowsfor more general predictions to be made. Additionally, the featurematrix 14 can be viewed as a complete profile of the network dynamicsover time, which can also find utility in other application settings ofevolutionary network analysis.

FIG. 3 illustrates link prediction in one or more sliding windows 200.

Regarding local random walk propagations 210, let an observed sequenceof temporal networks be G(t)=(N, A(t)), where N is a set of vertices,and A(t) is an adjacency matrix of the network at timestamp t∈[1, T],which is defined as a function of time.

The size of a vertex set is assumed to be |N|=n, and therefore A(t)∈

. The element a_(ijt) of A(t) is a link weight between vertices i and jat timestamp t.

The goal is to predict the links at timestamp T+α given A(1), A(2), . .. , A(T).

A label propagation principle is adopted, which states that two verticessimilar to each other are likely to have the same label.

A practical assumption can be made that two vertices (or nodes) that areconnected are likely to have similar features. Under this assumption,each vertex (or node) adjusts its feature vector based on its neighborsfeature vector.

Thus, suppose that an initial feature vector of vertex i is v_(i)(t) anda final state is f_(i)(t) at timestamp t. Then a propagation processfrom v_(i)(t) to f_(i)(t) can be modeled by the following optimizationproblem.

$\begin{matrix}{{\min\limits_{f{(t)}}{\lambda {\sum\limits_{i,j}{{A_{ij}(t)}{{{\frac{1}{\sqrt{D_{ii}(t)}}{f_{i}(t)}} - {\frac{1}{\sqrt{D_{jj}(t)}}{f_{j}(t)}}}}_{2}^{2}}}}} + {( {1 - \lambda} ){\sum\limits_{i}{{{f_{i}(t)} - {v_{i}(t)}}}^{2}}}} & (1)\end{matrix}$

where D(t)∈

is a degree matrix of A(t).

λ∈(0, 1) is a regularization weight.

The first term is the smoothness constraint 12, which enforcesneighboring vertices to have similar feature vectors. The second term isthe fitting constraint 10, which penalizes large deviation from theinitial feature vectors.

The above equation can be written in a matrix form as follows:

$\begin{matrix}{{\min\limits_{f{(t)}}{\lambda \; {f_{i}(t)}^{T}( {I - {\overset{\_}{A}(t)}} ){f_{i}(t)}}} + {( {1 - \lambda} ){{{f_{i}(t)} - {v_{i}(t)}}}_{F}^{2}}} & (2)\end{matrix}$

where I∈

is the identity matrix.

Ā(t) is the normalized version of A(t) which is defined as √{square rootover (D(t))}A(t)√{square root over (D(t))}.

The analytical solution of Eq. (2) is:

f _(i)(t)=(1−λ)(I−λĀ(t))⁻¹ v _(i)(t)  (3)

This explicit solution shows that the final feature vector is atransformation of the initial one based on the network structure attimestamp t.

Regarding the reconstruction error 220, f_(i)(t) is determined asfollows.

In the exemplary embodiments of the present invention, thetime-dependent matrix factorization method is leveraged, which naturallyexpresses the evolving network by learning a low rank representation ofthe underlying adjacency matrix.

The exemplary embodiments focus on undirected networks. However, oneskilled in the art can contemplate other types of networks, such as, butnot limited to, directed networks.

In this case, a symmetric adjacency matrix A(t) can be reconstructed bythe feature vectors {f_(i)({dot over (t)})}_(i=1) ^(n),

A(t)=F(t)F(t)^(T)  (4)

Here F(t)=[f₁, f₂, . . . , f_(n)]∈

is a time-depend feature matrix.

A standard approach is then followed to set up a least squaresoptimization problem so that A(t) and F(t)F{dot over (()}t)^(T) are asclose as possible.

This can be achieved by minimizing the Euclidean distance between allentries in A(t) and F(t)F{dot over (()}t)^(T).

Therefore, the optimization problem can be written as:

$\begin{matrix}{\min\limits_{F}{\sum\limits_{t = {\max {({1,{T - \omega}})}}}^{T}{\frac{h(t)}{2}{{{A(t)} - {{F(t)}{F(t)}^{T}}}}_{F}^{2}}}} & (5)\end{matrix}$

Here, h(t)=e^(−θ(T−t)) is an exponential decay function with time t thatregulates the importance of a current timestamp of a structured networkwith respect to the past or previous timestamps of structured networks.

ω is the sliding window size which only takes the recent ω timestampsinto consideration. This is pragmatic because there is no need to storeall network timestamps.

As can be seen from Eq. (5), in order to learn F(t), the time-dependentform of this matrix is regulated. For instance:

Let P(t)=(1−λ)(I−λĀ(t))−1.

Then F(t)=P(t)V(t) based on Eq. (3).

Subsequently, the optimization problem described by Eq. (5) has thefollowing form:

$\begin{matrix}{\min {\sum\limits_{t = {\max {({1,{T - \omega}})}}}^{T}{\frac{h(t)}{2}{{{A(t)} - {{P(t)}{V(t)}{V(t)}^{T}{P(t)}^{T}}}}_{F}^{2}}}} & (6)\end{matrix}$

Here V(t)∈

is a time-dependent matrix including all initial states of

vertex feature vectors. V(t)∈

characterizes the network dynamics by modeling the changes in a vertexfeature space. The function V(t) can take on any canonical form, such aslinear models, polynomial models, etc. based on the specific tasks. Apolynomial function is selected for V(t) because the method isattempting to fit the network dynamics within a small sliding window oflength ω.

Thus, V(t) can be represented as follows:

$\begin{matrix}{{V(t)} = {{W^{(0)} + {W^{(1)}t} + \ldots + {W^{(d)}t^{d}}} = {\sum\limits_{i = 0}^{d}{W^{(i)}t^{i}}}}} & (7)\end{matrix}$

Here {W^((i))}_(i=0) ^(d)∈

, d∈

.

V(t) is the simple linear function if d=1.

Regarding optimization 230, the challenge of optimizing the objectivefunction defined in Eq. (6) is that the network A(t) could be very largeand sparse, and the optimization of a O(n²) objective function is oftentoo computationally expensive.

Generally, the existence of a link provides more information than theabsence of a link which conveys far more noises. Therefore, Eq. (6)should be tuned up to focus on the non-zero entries in the adjacencymatrix A(t).

Let m be the number of non-zero entries in A(t). It is noted that asample of zero entries is still needed to properly train the model.

In the exemplary embodiments of the present invention, the sample sizeis set to be equal to the size of non-zero entries m.

Let S(t) be a sample indices at timestamp t such that a_(ijt)=0, ∀(i,j)∈S(t).

Let E(t) be a set of indices that need to be optimize in Eq. (6).

Then E(t)={(i, j)|a_(ijt)>0}∪S(t).

Note that a size of E(t) is much smaller than O(n²) because the networksare often very sparse.

Then the aforementioned objective function can be presented as follows:

$\begin{matrix}{\sum\limits_{t = {\max {({1,{T - \omega}})}}}^{T}{\frac{h(t)}{2}{\sum\limits_{{({i,j})} \in {E{(t)}}}( {a_{ijt} - ( {{P(t)}{V(t)}{V(t)}^{T}{P(t)}^{T}} )_{ij}} )^{2}}}} & (8)\end{matrix}$

For directed networks, A(t) can be decomposed into two matrices:

a constant matrix U and a time-dependent matrix V(t), both are n×kmatrices.

In this case, the optimization problem comes to be:

$\begin{matrix}{\sum\limits_{t = {\max {({1,{T - \omega}})}}}^{T}{\frac{h(t)}{2}{\sum\limits_{{({i,j})} \in {E{(t)}}}( {a_{ijt} - ( {{{UV}(t)}^{T}{P(t)}^{T}} )_{ij}} )^{2}}}} & (9)\end{matrix}$

Note that both U and V(t) can be selected to be time-dependent, but suchselection doubles the parameter space and increases the modelcomplexity.

Similar results can be achieved if one selects to make U time-dependentrather than V(t).

Model Learning

In this section, the algorithm to learn the LIST model is described. Theaforementioned objective function depends on a number of links in thenetworks.

Weight-decay terms also need to be added to reduce the variance of theLIST model.

Consider the objective function for undirected networks withweight-decay terms:

$\begin{matrix}{{J(W)} = {{\sum\limits_{t = {\max {({1,{T - \omega}})}}}^{T}{\frac{h(t)}{2}{{1_{E{(t)}}( {{A(t)} - {{P(t)}{V(t)}{V(t)}^{T}{P(t)}^{T}}} )}}_{F}^{2}}} + {\sum\limits_{i = 0}^{d}{\frac{\beta_{i}}{2}{W^{(i)}}_{F}^{2}}}}} & (10) \\{\mspace{79mu} {{where},}} & \; \\{\mspace{79mu} {{1_{E{(t)}}(M)} = \{ \begin{matrix}M_{ij} & {{{{if}\mspace{14mu} ( {i,j} )} \in {E(t)}},} \\0 & {{{if}\mspace{14mu} ( {i,j} )} \notin {{E(t)}.}}\end{matrix} }} & \;\end{matrix}$

{β_(i)}_(i=0) ^(d) are the weights of the weight-decay terms.

In order to infer parameter W, the symmetric matrix factorizationtechnique is leveraged to compute the derivatives of Eq. (10).

An “error term” ψ(t) is thus introduced for each timestamp t ofundirected networks as follows:

ψ(t)=1_(E(t))(A(t)−P(t)V(t)V(t)^(T) P(t)^(T))  (11)

Note that the error matrix in ψ(t) has already projected to indices setand defined by E(t).

Consider the derivative calculation of a simplified loss functionwithout timestamp t, then:

$\begin{matrix}{J = {\frac{1}{2}{{A - {{PVV}^{T}P^{T}}}}_{F}^{2}}} \\{= {\frac{1}{2}{{Tr}( {( {A - {{PVV}^{T}P^{T}}} )( {A - {{PVV}^{T}P^{T}}} )^{T}} )}}}\end{matrix}$ $\begin{matrix}{\frac{\partial J}{\partial V} = {{{- P^{T}}{APV}} - {P^{T}A^{T}{PV}} + {2P^{T}{PVV}^{T}P^{T}{PV}}}} \\{= {{- {P^{T}( {( {A - {{PVV}^{T}P^{T}}} ) + ( {A - {{PVV}^{T}P^{T}}} )^{T}} )}}{PV}}}\end{matrix}$

A common factor A−PVV^(T)P^(T) is extracted for each term in J.

The reason is that projection 1_(E(t))(⋅) can be applied on the commonfactor.

Therefore:

$\begin{matrix}{\frac{\partial{J(W)}}{\partial W^{(i)}} = {{\frac{\partial{J(W)}}{\partial{V(t)}}\frac{\partial{V(t)}}{\partial W^{(i)}}} = {{\sum\limits_{t = {\max {({1,{T - \omega}})}}}^{T}{{h(t)}{P(t)}^{T}( {{- {\psi (t)}} - {\psi (t)}^{T}} ){P(t)}{V(t)}t^{i}}} + {\beta_{i}W^{(i)}}}}} & (12)\end{matrix}$

The derivative over W needed to run a gradient descent is thus obtained.

The pseudocode for the LIST model is presented in Algorithm 1 below.

Algorithm 1: Algorithm for LIST model Input: temporal adjacency matrices {A(t)}_(t=max(1,T−w)) ^(T), the order d of V(t) and  latent dimension kOutput: factor matrices {W^((i))}_(i=1) ^(d) and the prediction A(T +1). Set k, d and w. Randomly initialize {W^((i))}_(i=0) ^(d). while notstopping criterion do $\quad\lfloor \begin{matrix}\begin{matrix}\begin{matrix}\begin{matrix}\begin{matrix}\begin{matrix}{{Compute}\mspace{14mu} {``{{error}\mspace{14mu} {term}}"}\mspace{14mu} {\psi (t)}\mspace{14mu} {for}\mspace{14mu} {each}\mspace{14mu} {time}\mspace{14mu} {stamp}\mspace{14mu} {t.}} \\{{{Compute}\mspace{14mu} {partial}\mspace{14mu} {derivatives}\mspace{14mu} \frac{\partial{J(W)}}{\partial W^{(i)}}\mspace{14mu} {using}\mspace{14mu} {\psi (t)}\mspace{14mu} {by}\mspace{14mu} {{Eq}.(12).}}\mspace{14mu}}\end{matrix} \\{{Determine}\mspace{14mu} {the}\mspace{14mu} {step}\mspace{14mu} {size}\mspace{14mu} \lambda \mspace{14mu} {by}\mspace{14mu} {line}\mspace{14mu} {{search}.}}\end{matrix} \\{{foreach}\mspace{14mu} i\mspace{14mu} {in}\mspace{14mu} \{ {1,\ldots \mspace{14mu},d} \} \mspace{14mu} {do}}\end{matrix} \\\lfloor \mspace{14mu} {{{Update}\mspace{14mu} W^{(i)}} = {W^{(i)} - {\lambda {\frac{\partial{J(W)}}{\partial W^{(i)}}.}}}} \end{matrix} \\{{{Compute}\mspace{14mu} {prediction}\mspace{14mu} {result}\mspace{14mu} {A( {T + 1} )}} =}\end{matrix} \\{( {\sum\limits_{i = 0}^{d}\; {W^{(i)}( {T + 1} )}^{i}} ){( {\sum\limits_{i = 0}^{d}\; {W^{(i)}( {T + 1} )}^{i}} )^{T}.}}\end{matrix} $

Computational Speed-Up

It is observed that an update of W(i) in Algorithm 1 computes an inverseof an n×n matrix I−λĀ(t) which runs in time O(n³).

In this section, an iterative method is used to approximate the matrixinverse calculation.

Given that eigenvalues of Q∈

in [−1,1], 0<γ<1, and an iteration number B, the inverse of matrix I−γQcan be approximated by summing up (γQ)^(b) across all iterations, whichis:

$\begin{matrix}{( {I - {\gamma \; Q}} )^{- 1} = {\lim\limits_{B->\infty}{\sum\limits_{b = 1}^{B}( {\gamma \; Q} )^{b - 1}}}} & (13)\end{matrix}$

Given λ∈(0,1) and Ā(t) (normalized by degree matrix D(t)), theapproximate solution for P(t) is:

$\begin{matrix}{{P(t)} = {{( {1 - \lambda} )( {I - {\lambda {\overset{\_}{A}(t)}}} )^{- 1}} = {( {1 - \lambda} ){\sum\limits_{b = 1}^{B}( {\lambda {\overset{\_}{A}(t)}} )^{b - 1}}}}} & (14)\end{matrix}$

Assume that the number of non-zero entries in the sparse matrix Ā(t) ism.

The complexity upper-bound of sparse matrix multiplication is O(mn).

Therefore, the above solution takes O((b−1)mn) in time for eachiteration, b∈[1, B], which results in an overall complexity O(B²mn) forthe calculation of P(t).

If the computed results in previously iterations are cached, then thetime complexity is reduced to O(Bmn).

Complexity Analysis

It is assumed that the gradient-descent method in Algorithm 1 isimplemented for M iterations, and the rank of the factorization is k.The bottleneck step is to update all parameters for d+1 matrices in eachof these iterations.

It can be seen from Eq. (12) that there are w timestamps within eachsliding window, and in each timestamp the complexity for matrixmanipulation is O(Bmn)+O(n²k).

Thus, the asymptotic running time is O(Mω(d+1)(Bmn+n²k)). In empiricalstudy settings, M, ω, d, B and k are much smaller than n and m,therefore the time complexity is approximately O(mn+n²).

The ability to predict link weights at a specific time is almost trivialusing the LIST model.

Once the factor matrices {W^((i))}_(i=1) ^(d), are obtained, thepredicted structure of the network can be effectively reconstructed byV(T+α)V(T+α)^(T) for any α≥1.

As the value of α becomes larger and larger, the reconstruction can beexpected to become increasingly challenging.

In conclusion, a link prediction model (LIST) has been developed fordynamic networks which simultaneously or concurrently incorporatesnetwork propagation and temporal matrix factorization techniques. Thisis guaranteed by the joint minimization of the network propagation lossand the temporal network reconstruction error. The LIST model utilizes auser-defined sliding window to learn the parameters, and thus supportsstreaming link prediction as well. The LIST model has the ability toexplicitly express the network as a function of time, which takes intoaccount both local (spatial) network structure and global evolving(temporal) patterns. Therefore, the LIST model has the advantage ofgenerality in addressing various temporal applications like temporalnetwork compression and expanding community detection.

FIG. 4 is a block/flow diagram illustrating a method for predictinglinks of a dynamic network, in accordance with embodiments of thepresent invention.

At block 301, link prediction in dynamic networks is employed. Linkprediction is employed with spatial and temporal consistency. Thetime-dependent matrix factorization technique is leveraged, which is apowerful tool for characterizing dynamic structural data to decomposenetwork adjacency matrices into time-dependent matrices that capture thefeatures of vertices (or nodes) in dynamic networks.

At block 303, a time-dependent matrix factorization method is employedto globally enforce the smoothness of network evolution over time. Thesmoothness constraint enforces neighboring vertices to have similarfeature vectors.

At block 305, the local network propagation constraints from the networktopology at each timestamp are utilized. The network propagationconstraint ensures vertices (or nodes) to be within close proximity totheir neighbors in the hidden feature space to be learned by thetime-dependent matrix factorization.

At block 307, an exponential decay function with time t is employed thatregulates an importance of a current timestamp with respect to previoustimestamps.

At block 309, a polynomial function is employed to fit the networkdynamics within a small sliding window of length ω. A polynomialfunction is selected for V(t) because the method is attempting to fitthe network dynamics within a small sliding window of length ω.

FIG. 5 is a block/flow diagram illustrating a method for learningfeature vectors of each node of a plurality of nodes of a sequence ofstructured networks by concurrently optimizing a temporal fittingconstraint and a network propagation constraint, in accordance withembodiments of the present invention.

At block 401, develop, at a plurality of timestamps, relational data ofa sequence of network structures each including a plurality of nodes.The relational data can be connections between the nodes (or vertices).A network is simply a collection of connected objects. The objects arereferred to as nodes or vertices, and are usually drawn as points. Theconnections between the nodes are referred to as edges, and are usuallydrawn as lines between points.

At block 403, learn, by the at least one processor, a feature vector ofeach node of the plurality of nodes of the sequence of networkstructures by concurrently optimizing a temporal fitting constraint anda network propagation constraint. The temporal fitting constraint isexpressed as a time-dependent matrix factorization with a networkadjacency matrix. The network propagation constraint maintainssimilarities between connected pairs of nodes in a feature space.

At block 405, determine a network structure at each timestamp of theplurality of timestamps. The learned feature vectors are parameterizedwith time and employed to reconstruct the network structure of thesequence of network structures at each of the plurality of timestamps.

At block 407, determine evolutionary patterns at each of the pluralityof timestamps.

At block 409, predict links in a future network structure based on anevolution of the sequence of network structures within a user-definedsliding window by reducing time complexities of finding neighbors ofeach node of the plurality of nodes of the sequence of networkstructures.

Networks can represent all sorts of systems in the real world. Forexample, the Internet can be described as a network where the nodes arecomputers or other devices and the edges are physical (or wireless,even) connections between the devices. The World Wide Web is a hugenetwork where the pages are nodes and links are the edges. Otherexamples include social networks of acquaintances or other types ofinteractions, networks of publications linked by citations,transportation networks, metabolic networks, communication networks, andInternet of Things (IoT) networks. The exemplary embodiments of thepresent invention can refer to any such networks without limitation.

FIG. 6 is an exemplary processing system for performing link predictionwith spatial and temporal consistency by employing a time-dependentmatrix factorization technique, in accordance with embodiments of thepresent invention.

The processing system includes at least one processor (CPU) 504operatively coupled to other components via a system bus 502. A cache506, a Read Only Memory (ROM) 508, a Random Access Memory (RAM) 510, aninput/output (I/O) adapter 520, a network adapter 530, a user interfaceadapter 540, and a display adapter 550, are operatively coupled to thesystem bus 502. Additionally, a dynamic network 501 and block 601 foremploying link prediction are operatively coupled to the system bus 502.The link prediction module 601 employs temporal fitting constraints 603and network propagation constraints 605.

A storage device 522 is operatively coupled to system bus 502 by the I/Oadapter 520. The storage device 522 can be any of a disk storage device(e.g., a magnetic or optical disk storage device), a solid statemagnetic device, and so forth.

A transceiver 532 is operatively coupled to system bus 502 by networkadapter 530.

User input devices 542 are operatively coupled to system bus 502 by userinterface adapter 540. The user input devices 542 can be any of akeyboard, a mouse, a keypad, an image capture device, a motion sensingdevice, a microphone, a device incorporating the functionality of atleast two of the preceding devices, and so forth. Of course, other typesof input devices can also be used, while maintaining the spirit of thepresent invention. The user input devices 542 can be the same type ofuser input device or different types of user input devices. The userinput devices 542 are used to input and output information to and fromthe processing system.

A display device 552 is operatively coupled to system bus 502 by displayadapter 550.

Of course, the dynamic network processing system may also include otherelements (not shown), as readily contemplated by one of skill in theart, as well as omit certain elements. For example, various other inputdevices and/or output devices can be included in the system, dependingupon the particular implementation of the same, as readily understood byone of ordinary skill in the art. For example, various types of wirelessand/or wired input and/or output devices can be used. Moreover,additional processors, controllers, memories, and so forth, in variousconfigurations can also be utilized as readily appreciated by one ofordinary skill in the art. These and other variations of the dynamicnetwork processing system are readily contemplated by one of ordinaryskill in the art given the teachings of the present invention providedherein.

FIG. 7 is a block/flow diagram of a method for performing linkprediction in Internet of Things (IoT) systems/devices/infrastructure,in accordance with embodiments of the present invention.

IoT enables advanced connectivity of computing and embedded devicesthrough internet infrastructure. IoT involves machine-to-machinecommunications (M2M), where it is important to continuously monitorconnected machines to detect any anomaly or bug, and resolve themquickly to minimize downtime.

The LIST model 601 can be incorporated, e.g., into wearable,implantable, or ingestible electronic devices and Internet of Things(IoT) sensors. The wearable, implantable, or ingestible devices caninclude at least health and wellness monitoring devices, as well asfitness devices. The wearable, implantable, or ingestible devices canfurther include at least implantable devices, smart watches,head-mounted devices, security and prevention devices, and gaming andlifestyle devices. The IoT sensors can be incorporated into at leasthome automation applications, automotive applications, user interfaceapplications, lifestyle and/or entertainment applications, city and/orinfrastructure applications, toys, healthcare, fitness, retail tagsand/or trackers, platforms and components, etc. The LIST model or module601 described herein can be incorporated into any type of electronicdevices for any type of use or application or operation.

IoT (Internet of Things) is an advanced automation and analytics systemwhich exploits networking, sensing, big data, and artificialintelligence technology to deliver complete systems for a product orservice. These systems allow greater transparency, control, andperformance when applied to any industry or system.

IoT systems have applications across industries through their uniqueflexibility and ability to be suitable in any environment. IoT systemsenhance data collection, automation, operations, and much more throughsmart devices and powerful enabling technology.

IoT systems allow users to achieve deeper automation, analysis, andintegration within a system. IoT improves the reach of these areas andtheir accuracy. IoT utilizes existing and emerging technology forsensing, networking, and robotics. Features of IoT include artificialintelligence, connectivity, sensors, active engagement, and small deviceuse. In various embodiments, the LIST model 601 of the present inventioncan be incorporated into a variety of different devices and/or systems.For example, the LIST model 601 can be incorporated into wearable orportable electronic devices 830. Wearable/portable electronic devices830 can include implantable devices 831, such as smart clothing 832.Wearable/portable devices 830 can include smart watches 833, as well assmart jewelry 834. Wearable/portable devices 830 can further includefitness monitoring devices 835, health and wellness monitoring devices837, head-mounted devices 839 (e.g., smart glasses 840), security andprevention systems 841, gaming and lifestyle devices 843, smartphones/tablets 845, media players 847, and/or computers/computingdevices 849.

The LIST model 601 of the present invention can be further incorporatedinto Internet of Thing (IoT) sensors 810 for various applications, suchas home automation 821, automotive 823, user interface 825, lifestyleand/or entertainment 827, city and/or infrastructure 829, retail 811,tags and/or trackers 813, platform and components 815, toys 817, and/orhealthcare 819. The IoT sensors 810 can communicate with the LIST model601. Of course, one skilled in the art can contemplate incorporatingsuch LIST model 601 formed therein into any type of electronic devicesfor any types of applications, not limited to the ones described herein.

FIG. 8 is a block/flow diagram of exemplary IoT sensors used to collectdata/information related to performing link prediction, in accordancewith embodiments of the present invention.

IoT loses its distinction without sensors. IoT sensors act as defininginstruments which transform IoT from a standard passive network ofdevices into an active system capable of real-world integration.

The IoT sensors 810 can be connected via LIST model 601 to transmitinformation/data, continuously and in real-time, to any type of dynamicnetwork 501. Exemplary IoT sensors 810 can include, but are not limitedto, position/presence/proximity sensors 901, motion/velocity sensors903, displacement sensors 905, such as acceleration/tilt sensors 906,temperature sensors 907, humidity/moisture sensors 909, as well as flowsensors 910, acoustic/sound/vibration sensors 911, chemical/gas sensors913, force/load/torque/strain/pressure sensors 915, and/orelectric/magnetic sensors 917. One skilled in the art can contemplateusing any combination of such sensors to collect data/information andinput into the LIST model 601 for further processing. One skilled in theart can contemplate using other types of IoT sensors, such as, but notlimited to, magnetometers, gyroscopes, image sensors, light sensors,radio frequency identification (RFID) sensors, and/or micro flowsensors. IoT sensors can also include energy modules, power managementmodules, RF modules, and sensing modules. RF modules managecommunications through their signal processing, WiFi, ZigBee®,Bluetooth®, radio transceiver, duplexer, etc.

Moreover data collection software can be used to manage sensing,measurements, light data filtering, light data security, and aggregationof data. Data collection software uses certain protocols to aid IoTsensors in connecting with real-time, machine-to-machine networks. Thenthe data collection software collects data from multiple devices anddistributes it in accordance with settings. Data collection softwarealso works in reverse by distributing data over devices. The system caneventually transmit all collected data to, e.g., a central server.

FIG. 9 is a block/flow diagram illustrating a completed dynamic networkby using a link prediction model with spatial and temporal consistency(LIST), in accordance with embodiments of the present invention.

An initial network 950 includes, e.g., 5 nodes. Nodes 1 and 2 areconnected, and nodes 2 and 5 are connected. This initial state isdesignated as T=0. The LIST model is applied to the initial network 950.As a result of this link prediction iteration, another link ispredicted, that is, a connection link between nodes 2 and 4. This stateis designated as T=1 and the new network is designated as dynamicnetwork 960. The LIST model is now applied to the new network 960. As aresult of this link prediction iteration, another link is predicted,that is, a connection link between nodes 2 and 3. This state isdesignated as T=2 and the new network is designated as dynamic network970. The LIST model is now applied to the new dynamic network 970. As aresult of this link prediction iteration, more links are predicted, thatis, a connection link between nodes 1 and 5, 4 and 5, and 3 and 4. Thisstate is designated as T=3 and is the final state where the finaldynamic network 980 is created. Thus, dynamic network 980 is createdafter a number of iterations using the LIST model. Predictions occur atevery stage or state to predict new possible links between nodes of adynamic network. The temporal fitting constraint 10 and the propagationconstraint 12 are applied at each state (FIG. 1) according to the LISTmodel. Dynamic network 980 is a network that varies over time and hasvertices that are often not binary and instead represent a probabilityfor having a link between at least two nodes. Statistical approaches orcomputer simulations are often necessary to explore how such dynamicnetworks evolve, adapt or respond to external intervention. The LISTmodel aids in the evolution of dynamic network 980 as presented in atleast FIG. 9.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical datastorage device, a magnetic data storage device, or any suitablecombination of the foregoing. In the context of this document, acomputer readable storage medium may be any tangible medium that caninclude, or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thepresent invention. It will be understood that each block of theflowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks or modules.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks or modules.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks or modules.

It is to be appreciated that the term “processor” as used herein isintended to include any processing device, such as, for example, onethat includes a CPU (central processing unit) and/or other processingcircuitry. It is also to be understood that the term “processor” mayrefer to more than one processing device and that various elementsassociated with a processing device may be shared by other processingdevices.

The term “memory” as used herein is intended to include memoryassociated with a processor or CPU, such as, for example, RAM, ROM, afixed memory device (e.g., hard drive), a removable memory device (e.g.,diskette), flash memory, etc. Such memory may be considered a computerreadable storage medium.

In addition, the phrase “input/output devices” or “I/O devices” as usedherein is intended to include, for example, one or more input devices(e.g., keyboard, mouse, scanner, etc.) for entering data to theprocessing unit, and/or one or more output devices (e.g., speaker,display, printer, etc.) for presenting results associated with theprocessing unit.

The foregoing is to be understood as being in every respect illustrativeand exemplary, but not restrictive, and the scope of the inventiondisclosed herein is not to be determined from the Detailed Description,but rather from the claims as interpreted according to the full breadthpermitted by the patent laws. It is to be understood that theembodiments shown and described herein are only illustrative of theprinciples of the present invention and that those skilled in the artmay implement various modifications without departing from the scope andspirit of the invention. Those skilled in the art could implementvarious other feature combinations without departing from the scope andspirit of the invention. Having thus described aspects of the invention,with the details and particularity required by the patent laws, what isclaimed and desired protected by Letters Patent is set forth in theappended claims.

What is claimed is:
 1. A computer-implemented method executed by atleast one processor for performing link prediction with spatial andtemporal consistency by employing a time-dependent matrix factorizationtechnique, the method comprising: developing, at a plurality oftimestamps, relational data of a sequence of network structures eachincluding a plurality of nodes; learning, by the at least one processor,a feature vector of each node of the plurality of nodes of the sequenceof network structures by concurrently optimizing a temporal fittingconstraint and a network propagation constraint; determining a networkstructure at each timestamp of the plurality of timestamps; determiningevolutionary patterns at each of the plurality of timestamps; andpredicting links in a future network structure based on an evolution ofthe sequence of network structures within a user-defined sliding windowby reducing time complexities of finding neighbors of each node of theplurality of nodes of the sequence of network structures.
 2. The methodof claim 1, further comprising expressing the temporal fittingconstraint as a time-dependent matrix factorization with a networkadjacency matrix.
 3. The method of claim 2, further comprisingmaintaining similarities between connected pairs of nodes in a featurespace via the network propagation constraint.
 4. The method of claim 3,further comprising parameterizing the learned feature vectors with timeand employing the learned feature vectors to reconstruct the networkstructure of the sequence of network structures at each of the pluralityof timestamps.
 5. The method of claim 1, further comprising adjustingthe feature vector of each node of the plurality of nodes of thesequence of network structures based on a feature vector of aneighboring node.
 6. The method of claim 1, further comprisingexpressing the future network structure as a function of time.
 7. Themethod of claim 1, further comprising determining, via an exponentialdecay function, an importance of a current timestamp of a currentstructure network with respect to previous timestamps of previousstructure networks.
 8. The method of claim 1, further comprisingdetermining the user-defined sliding window based on a polynomialfunction.
 9. A system including a dynamic network for performing linkprediction with spatial and temporal consistency by employing atime-dependent matrix factorization technique, the system comprising: amemory; and a processor in communication with the memory, wherein theprocessor is configured to: develop, at a plurality of timestamps,relational data of a sequence of network structures each including aplurality of nodes; learn, by the at least one processor, a featurevector of each node of the plurality of nodes of the sequence of networkstructures by concurrently optimizing a temporal fitting constraint anda network propagation constraint; determine a network structure at eachtimestamp of the plurality of timestamps; determine evolutionarypatterns at each of the plurality of timestamps; and predict links in afuture network structure based on an evolution of the sequence ofnetwork structures within a user-defined sliding window by reducing timecomplexities of finding neighbors of each node of the plurality of nodesof the sequence of network structures.
 10. The system of claim 9,wherein the temporal fitting constraint is expressed as a time-dependentmatrix factorization with a network adjacency matrix.
 11. The system ofclaim 10, wherein the network propagation constraint maintainssimilarities between connected pairs of nodes in a feature space. 12.The system of claim 11, wherein the learned feature vectors areparameterized with time and employed to reconstruct the networkstructure of the sequence of network structures at each of the pluralityof timestamps.
 13. The system of claim 9, wherein the feature vector ofeach node of the plurality of nodes of the sequence of networkstructures is adjusted based on a feature vector of a neighboring node.14. The system of claim 9, wherein the future network structure isexpressed as a function of time.
 15. The system of claim 9, wherein anexponential decay function determines an importance of a currenttimestamp of a current structure network with respect to previoustimestamps of previous structure networks.
 16. The system of claim 9,wherein the user-defined sliding window is determined based on apolynomial function.
 17. A non-transitory computer-readable storagemedium comprising a computer-readable program for performing linkprediction with spatial and temporal consistency by employing atime-dependent matrix factorization technique, wherein thecomputer-readable program when executed on a computer causes thecomputer to perform the steps of: developing, at a plurality oftimestamps, relational data of a sequence of network structures eachincluding a plurality of nodes; learning, by the at least one processor,a feature vector of each node of the plurality of nodes of the sequenceof network structures by concurrently optimizing a temporal fittingconstraint and a network propagation constraint; determining a networkstructure at each timestamp of the plurality of timestamps; determiningevolutionary patterns at each of the plurality of timestamps; andpredicting links in a future network structure based on an evolution ofthe sequence of network structures within a user-defined sliding windowby reducing time complexities of finding neighbors of each node of theplurality of nodes of the sequence of network structures.
 18. Thenon-transitory computer-readable storage medium of claim 17, wherein thetemporal fitting constraint is expressed as a time-dependent matrixfactorization with a network adjacency matrix.
 19. The non-transitorycomputer-readable storage medium of claim 18, wherein the networkpropagation constraint maintains similarities between connected pairs ofnodes in a feature space.
 20. The non-transitory computer-readablestorage medium of claim 19, wherein the future network structure isexpressed as a function of time; and wherein the user-defined slidingwindow is determined based on a polynomial function.